1. Field of the Invention
The present invention relates to an image processing technique including image deformation processing.
2. Description of the Related Art
In a product such as a projector, when light output from the projector is projected on a screen, the effective region projected on the screen is distorted into a trapezoidal shape due to the installation tilt angle of the projector, an optical lens shift, and the like. It is difficult for the user to view the image without eliminating the trapezoid distortion. Hence, image deformation processing of deforming the effective region into an inverted trapezoidal shape is generally performed so that the effective region projected on the screen obtains a rectangular shape. This image deformation processing is known as, for example, keystone distortion correction processing or keystone correction processing.
The resolutions of videos are rising in recent years, and a higher-resolution video needs to be processed by the above-described image deformation processing. However, an image deformation processing apparatus normally has an upper limit to the processable resolution and may be unable to process a required resolution on a standalone basis. The above-described upper limit of the processable resolution is decided by processing throughput attributed to the operating frequency of the image deformation processing apparatus, the horizontal-direction size of the line buffer of an internal buffer included as a constituent element, or the transfer band of the DRAM of a frame memory included as a constituent element.
To solve the above-described problem, related arts have been proposed by, for example, patent literature 1 (Japanese Patent Laid-Open No. 2008-312099) and patent literature 2 (Japanese Patent No. 3735158). In these methods, an input video having a high resolution is cut out to a plurality of images, and a plurality of image deformation processing apparatuses process the respective images, thereby solving the above-described problem.
More specifically, in the method described in patent literature 1, the apparatus includes a plurality of image cutout units that cut out divided images from an original image within the range in which no pixel losses occur at the boundaries of the divided images synthesized on the projector. The apparatus also includes a plurality of image deformation units that deform and correct the divided images cut out by the plurality of image cutout units in accordance with the three-dimensional shape of the projection plane so as not to distort the projected image, and output the images to the projector. Note that only characteristic elements have selectively been described here for the sake of simplicity.
The system described in patent literature 2 is an image projection system that arranges projection images from a plurality of projectors while making them overlap each other and displays one high-resolution image corresponding to input high-resolution image data on the screen. This system includes an image capture unit that captures a reference image projected on the screen by inputting reference image data to each projector. The system refers to parameters concerning the position of the projection region of each projector on the screen, which are calculated and stored based on the reference image capture data obtained by the image capture unit. The system also includes an image processing unit that cuts out, from the high-resolution image data, an image region corresponding to the position of each projection region based on the referred parameters and outputs the image to each projector.
As described above, both the related arts of patent literatures 1 and 2 implement distribution of processing and solve the above-described problem.
In the image deformation processing, the size of the effective region of an image changes between input and output. For this reason, in the image deformation apparatus including the plurality of image deformation units, it may be necessary to make the input image size larger than the output image size in the individual image deformation units.
FIG. 1 shows this state. In FIG. 1, the left area of an input image (All input area) 100 having a horizontal size in_w_all (ot_w_all) and a vertical size in_h_all (ot_h_all) is deformed by a left image deformation unit, and the right area is deformed by a right image deformation unit. As a result, an output image (All output area) 101 is obtained by deforming the input image 100.
In keystone distortion correction processing of a projector product, the deformed shape of the output image 101 is decided by parameters such as the projection distance from the projector to the screen, the vertical/horizontal installation angles, and the vertical/horizontal lens shifts.
The respective image deformation units have the same horizontal output size ot_w_left=ot_w_right=(ot_w_all/2). The image deformation units also have the same vertical input and output sizes in_h_all=ot_h_all.
The left image deformation unit will be described in particular. To output the horizontal output size ot_w_left, the image deformation unit needs to output a trapezoid area 103 indicated by the white area in the left output area. Since the trapezoid area 103 corresponds to a rectangle area 102 indicated by the white area in the left input area, a horizontal input size in_w_left (>in_w_right) necessary for the image deformation unit can be calculated. This also applies to the right image deformation unit. As a result, the horizontal input sizes in_w_left and in_w_right to be input to the individual image deformation units are known.
The input sizes processable by the individual image deformation units normally have an upper limit. The upper limit is decided by processing throughput attributed to the operating frequency of the image deformation processing apparatus, an internal buffer included as a constituent element (for example, the horizontal-direction size of a line buffer), a frame memory included as a constituent element (for example, the transfer band of a DRAM), or the like.
For this reason, the horizontal input sizes in_w_left and in_w_right to the individual image deformation processes need to be equal to or smaller than the upper limit value of the horizontal input sizes. However, if the effective region of the output image is uneven in the deformation, the horizontal input sizes in_w_left and in_w_right to the individual image deformation units also have unevenness. As a result, the horizontal input sizes exceed the upper limit even if the degree of deformation is small.
Whether the above-described problem can arise in the related arts mentioned in “Description of the Related Art” will be described next. Patent literature 1 describes no method of deciding the effective region of the output image. The description is done by exemplifying a case in which the effective region of the output image has an arbitrary shape. In this method, however, there is no configuration for correcting the position of the effective region of the output image in accordance with the limit of the individual image deformation units. For this reason, the unevenness of the effective region of the output image cannot be averaged. As a result, it can be said that the above-described problem that the horizontal input sizes exceed the upper limit value even if the degree of deformation is small may arise.
Patent literature 2 describes, as the method of deciding the effective region of the output image, a method of deciding the effective region of the output image using, as the reference point, the vertex having the largest angle out of the vertices of the tetragon projected on the screen. More specifically, out of the vertices of a tetragon indicating the maximum range of a projected image represented by “the projected image in FIG. 4(a)” of patent literature 2, “PO” having the largest angle is used as the reference point. However, this method cannot average the unevenness of the effective region of the output image. As a result, it can be said that the above-described problem that the horizontal input sizes exceed the upper limit value even if the degree of deformation is small may arise.
In addition, the input image before division cannot completely be displayed in some cases. More specifically, the lower portion of the left-side image or the upper portion of the right-side image of “the image projected on the screen shown in FIG. 3(d)” of patent literature 2 is not displayed. If this phenomenon occurs, it may be impossible to satisfy the necessary conditions at all.
As described above, in the related arts, the degree of deformation is largely limited because of the unevenness of the effective region of the output image.